Optical sensor arrangement and method for gesture detection

ABSTRACT

An optical sensor arrangement ( 10 ) comprises at least two light sensors ( 35, 36 ) for providing at least two sensor signals (S 1 , S 2 ), and an evaluation circuit ( 12 ) that comprises an input ( 16 ) coupled to the at least two light sensors ( 35, 36 ) and is designed to evaluate the at least two sensor signals (S 1 , S 2 ) for gesture detection in an adaptive manner using previous values of the at least two sensor signals (S 1 , S 2 ).

BACKGROUND OF THE INVENTION

The present invention is related to an optical sensor arrangement and amethod for gesture detection.

Some gesture technologies require touching a screen or physically movinga mouse device. As an alternative touchless gesture mechanisms aredeveloped. For example, cameras are being used to detect human gestures.However, these systems are expensive and complex and only work at longdistances.

Document US 2012/0280904 A1 describes a method for detecting gesturesusing a multi-segment photodiode.

SUMMARY OF THE INVENTION

In an embodiment, an optical sensor arrangement comprises at least twolight sensors for providing at least two sensor signals and anevaluation circuit that comprises an input coupled to the at least twolight sensors. The evaluation circuit is designed to evaluate the atleast two sensor signals for gesture detection in an adaptive mannerusing previous values of the at least two sensor signals.

Since the at least two sensor signals not only depend on a gesture butalso to a changing surrounding, an adaptive evaluation on the basis ofprevious values of the at least two sensor signals improves therecognition of the gesture.

In an embodiment, the evaluation circuit is designed to start capturingthe at least two sensor signals for gesture detection, if a signalderived from the at least two sensor signals is larger than a gestureentry threshold. Furthermore, the evaluation circuit is designed todynamically adapt the gesture entry threshold depending on the previousvalues of the at least two sensor signals.

Advantageously, the gesture entry threshold is adapted dynamically as afunction of the previous values of the at least two sensor signals.Thus, the influence of ambient light and other objects near the opticalsensor arrangement is reduced. The at least two captured sensors signalsare optionally interpreted for gesture detection.

In an embodiment, the evaluation circuit is designed to determine thegesture entry threshold by adding a distance signal to a baselinesignal.

Thus, the gesture entry threshold can advantageously be determined by aflexible use of at least two parameters.

In an embodiment, the evaluation circuit is designed to determine anoise signal by noise filtering of a sum of the at least two sensorsignals. The distance signal depends on the noise signal.

Advantageously, the gesture entry threshold is increased in case ofnoise in the previous values of the at least two sensor signals.

In a further development, the evaluation circuit is designed todetermine the distance signal by multiplying a standard deviation factorwith the noise signal.

In an embodiment, the evaluation circuit is designed to determine a slowfilter signal by filtering the sum of the at least two sensor signalswith a slow filter, a medium filter signal by filtering the sum of theat least two sensor signals with a medium filter, and the baselinesignal by selecting the minimum of the slow filter signal and the mediumfilter signal.

In an embodiment, a cutoff frequency of the slow filter is lower than acutoff frequency of the medium filter. The cutoff frequency may also benamed corner frequency or break frequency. The slow and the mediumfilter are realized as low pass filters. The slow filter signal islimited to a smaller cutoff frequency than the medium filter signal.

In an embodiment, the evaluation circuit is designed to stop capturingthe at least two sensor signals for gesture detection, if the signalderived from the at least two sensor signals is lower than a gestureexit threshold. The gesture exit threshold may be equal to the gestureentry threshold.

In a further development, the evaluation circuit is designed todetermine the gesture exit threshold by generating a first multiplieddistance signal by multiplying the distance signal with a firstpredetermined factor and adding the first multiplied distance signal andthe gesture entry threshold.

In an embodiment, the optical sensor arrangement is designed to generateat least two filtered signals by filtering the at least two sensorsignals and to generate at least two offset-compensated sensor signalsby subtracting the filtered signals from the corresponding sensorsignals.

Advantageously, each of the sensor signals receive an offsetcompensation. Contrary to previous methods wherein the offset is notreduced or is reduced only by a predetermined amount, the offset is nowcontrolled in an adaptive manner. Without offset adaptation, the offsetcould be either very positive or negative. If negative, it would greatlyreduce the maximum distance the optical sensor arrangement could sense.If positive, it would affect the dynamic range of the device.

In an embodiment, the evaluation circuit is designed to generate atleast two scaled sensor signals by multiplying the at least twooffset-compensated sensor signals with at least two scaling factors orby adding the at least two offset-compensated sensor signals and atleast two scaling coefficients.

In an embodiment, the signal derived from the at least two sensorsignals which are compared with the gesture entry threshold for startingcapturing the at least two sensor signals is one of the scaled sensorsignals or the sum of more than one of the scaled sensor signals.

In an embodiment, an optical sensor arrangement comprises at least twolight sensors for providing at least two sensor signals, and anevaluation circuit. The evaluation circuit comprises an input coupled tothe at least two light sensors. The evaluation circuit is designed togenerate an entry vector indicating the position of an object relativeto the at least two light sensors at the start of a gesture and an exitvector indicating the position of the object relative to the at leasttwo light sensors at the end of the gesture.

By the determination of the entry vector and the exit vector, a gesturecan be analyzed more thoroughly and therefore more information can begained from a gesture. Whereas previous methods allow for only a singlevector of a gesture detection, now entry and exit vectors advantageouslydetermine a gesture. The use of entry and exit vectors combined withcareful analysis allows more gestures to be decoded, thus increasing thegesture vocabulary of the optical sensor arrangement.

In an embodiment, the first coordinate of a vector is a north to southstrength and the second coordinate of the vector is an east to weststrength. This is valid for the entry and the exit vector. Thus, thefirst coordinate of the entry vector and of the exit vector is the northto south strength at the start and at the end of the gesture,respectively. The second coordinate of the entry vector and of the exitvector is the east to west strength at the start and at the end of thegesture, respectively.

In an embodiment, the values of the first coordinate of the entry vectorand of the exit vector may be different or equal. Correspondingly, thevalues of the second coordinate of the entry vector and of the exitvector may be different or equal. By the optical sensor arrangementeight basic functions can be decoded such as the directions north tosouth, south to north, east to west, west to east, northeast tosouthwest, southwest to northeast, northwest to southeast and southeastto northwest. Using more complex decoding, eight additional functionscan be decoded including north tap, south tap, east tap, west tap, northbutton push, south button push, east button push and west button push.Whereas south tap is a short gesture, south button push would is a longgesture. North, south, west and east are equivalents to top, bottom,left and right or y, −y, x and −x direction.

In an embodiment, the evaluation circuit is designed to recognize a tapgesture, if the entry vector and the exit vector are nearly the same andthe duration of the gesture is less than a first predetermined duration.

In an embodiment, the evaluation circuit is designed to recognize abutton gesture, if the entry vector and the exit vector are nearly thesame and the duration of the gesture is larger than a secondpredetermined duration.

In an embodiment, a method for gesture detection comprises providing atleast two sensor signals by at least two light sensors. Moreover, the atleast two sensor signals are evaluated for gesture detection in anadaptive manner using previous values of the at least two sensorsignals.

In a further development, a gesture entry threshold is dynamicallyadapted depending on the previous values of the at least two sensorsignals. Furthermore, the at least two sensor signals are captured forgesture detection, if a signal derived from the at least two sensorsignals is larger than the gesture entry threshold.

Whereas previous methods had a fixed threshold above which the sensorresponse would be considered valid, the threshold advantageously is nowadaptive. Without threshold adaptation, the distance sensitivity wouldvary from device to device and gestures would only be detected at a muchshorter distance.

The method is performed on-line. The method steps such as the adaptingof the gesture entry threshold and the capturing of the at least twosensor signals as well as other method steps may be performed by anevaluation circuit on-line.

In an embodiment, the method for gesture detection comprises generatingat least two filtered signals by filtering the at least two sensorsignals. At least two offset-compensated sensor signals are generated bysubtracting the filtered signals from the corresponding sensor signals.

In an embodiment, a method for gesture detection comprises providing atleast two sensor signals by at least two light sensors, and generatingan entry vector indicating the position of an object relative to the atleast two light sensors at the start of a gesture and an exit vectorindicating the position of the object relative to the at least two lightsensors at the end of the gesture.

The optical sensor arrangement can be applied for touchless gesturesensing in a computer system or a device for mobile communication. As aresult of advances in silicon technology costs can be substantiallylowered. Alternatively, the optical sensor arrangement can be used inhouse thermostats and in the medical field. Doctors no longer need totouch anything in order to interact with a computer, thus eliminatingpotential contact with germs on surfaces.

In a further development, the evaluation circuit with the adaptivethreshold and/or the offset control can also be designed to generate theentry vector and the exit vector indicating the position of the objectrelative to the at least two light sensors at the start and at the endof the gesture.

Thus, the previously described methods for gesture detection can becombined. The evaluation circuit can be designed to implement a singleone or two or more of the methods.

In an embodiment, capturing the at least two sensor signals starts atthe start of the gesture. Thus, the entry vector may be determined, whenthe signal derived from the at least two sensor signals is larger thanthe gesture entry threshold.

Correspondingly, capturing the at least two sensor signals may end atthe end of the gesture. Thus, the exit vector may be determined, whenthe signal derived from the at least two sensor signals is lower thanthe gesture exit threshold.

Optionally, the optical sensor arrangement relates to light sensors, andmore specifically to proximity and gesture sensing. The optical sensorarrangement may provide dynamic threshold calibration to allow forproximity and gesture sensing at the longest possible distance. Theoptical sensor arrangement may combine methods and algorithm foradaptive threshold, offset control and the use of entry and exit vectorsin gesture sensing.

In an embodiment, the optical sensor arrangement obtains two or moredirectional components.

The at least two light sensors may be sensitive in different directions.

In an embodiment, the optical sensor arrangement comprises a lightsource. The at least two light sensors and the light source are packagedin the same subsystem. The at least two light sensors detect reflectedlight which is emitted by the light source. The reflected light may beinfrared light. The light sensors may be implemented as passive infraredsensors, abbreviated PIR sensors. Thus, the light sensors can also berealized as heat sensors.

In an embodiment, the optical sensor arrangement is sensitive in twodirectional components. Thus, the optical sensor arrangement comprisesexactly two light sensors.

In an alternative embodiment, the optical sensor arrangement issensitive in four directional components. Thus, the optical sensorarrangement comprises exactly four light sensors.

In an alternative embodiment, the optical sensor arrangement is free ofan electric light source. Instead, the optical sensor arrangement usesambient light as light source.

In an embodiment, the optical sensor arrangement is configured forgesture detection. The at least two sensor signals are evaluated forgesture detection. Gesture detection can also be named gesture sensingor gesture recognition.

In a further embodiment, the optical sensor arrangement is configuredfor proximity and gesture detection. The at least two sensor signals areevaluated for proximity and gesture detection. Proximity and gesturedetection can also be named proximity and gesture sensing.

In an embodiment, the optical sensor arrangement operates in anon-contact mode. The at least two light sensors operate in thenon-contact mode. The at least two light sensors detect non-contactgestures.

In an embodiment, the at least two light sensors are not comprised by acamera. The optical sensor arrangement is free of a camera.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description of figures of exemplary embodiments mayfurther illustrate and explain the invention. In so far as components,devices and method steps correspond to one another in terms of theirfunction in different figures, the description thereof is not repeatedfor each of the following figures.

FIGS. 1A to 1D show exemplary embodiments of an optical sensorarrangement and signal characteristics;

FIGS. 2A to 2J show an exemplary embodiment of adaptive filters and anenter/exit thresholds process as well as corresponding signalcharacteristics;

FIGS. 3A to 3C show an exemplary embodiment of crosstalk filters andcorresponding signal characteristics;

FIGS. 4A to 4C show an alternative embodiment of the crosstalk filtersand corresponding signal characteristics;

FIGS. 5A to 5C show a further exemplary embodiment of the optical sensorarrangement; and

FIGS. 6A to 6K show an exemplary embodiment of a gesture messageprocessing and corresponding signal characteristics.

DETAILED DESCRIPTION

FIG. 1A shows an exemplary embodiment of an optical sensor arrangement10. The optical sensor arrangement 10 comprises an input module 11 andan evaluation circuit 12. The input module 11 comprises a photodiodeunit 13, an analog-to-digital converter 14 and a memory 15. Thephotodiode unit 13 comprises at least two light sensors 35, 36. Theanalog-to-digital converter 14, abbreviated ADC, couples the photodiodeunit 13 to the memory 15. The memory 15 is realized as a First In FirstOut memory, abbreviated FIFO. The memory 15 comprises, for example, 128bytes. An output of the memory 15 is connected to an output of the inputmodule 11. The output of the input module 11 is coupled to an input 16of the evaluation circuit 12. For the coupling between the input module11 and the evaluation circuit 12 an Inter-Integrated Circuit bus,abbreviated I2C bus is used. The evaluation circuit 12 comprises asummation step 17, adaptive filters 18, an entry/exit thresholds process19, crosstalk filters 20, an acquisition process 21, a gesture messageprocessing 22, a gesture event processing 23 and a gesture gameprocessing 24.

The evaluation circuit 12 is realized as a microprocessor. Therefore,the filters, blocks, units, steps and processes 17 to 24 of theevaluation circuit 12 shown in FIG. 1A and the following figures areimplemented by processes or steps of a software which run on themicroprocessor of the evaluation circuit 12. The microprocessor performsthe processes and steps on-line.

The at least two light sensors 35, 36 generate at least two analogsensor signals AS1, AS2. The ADC 14 converts the at least two analogsensor signals AS1, AS2 into at least two sensor signals 51, S2 indigital form. The at least two sensor signals S1, S2 are stored in thememory 15. The input module 11 provides the at least two sensor signalsS1, S2 to the input 16 of the evaluation circuit 12. The at least twosensor signals S1, S2 are summed by the summation step 17 resulting in asumming signal SU. The summing signal SU is provided to the adaptivefilters 18. Output signals SN, SB of the adaptive filters 18 areprovided to the entry/exit thresholds process 19 for generating agesture entry threshold GEN and a gesture exit threshold GEX.Furthermore, the output signals SB, SN of the adaptive filters 18 areused by the enter/exit thresholds process 19 to provide a runtimegesture threshold RGT.

The at least two sensor signals S1, S2 are provided to the crosstalkfilters 20. The crosstalk filters 20 generate at least two scaled sensorsignals SC1, SC2 which are provided to the acquisition process 21. Ifone of the at least two scaled sensor signals SC1, SC2 is larger thanthe gesture entry threshold GEN, than the at least two scaled sensorsignals SC1, SC2 are captured and further processed by the gesturemessage processing 22, the gesture event processing 23 and/or thegesture game processing 24. If each of the at least two scaled sensorsignals SC1, SC2 is smaller than the gesture exit threshold GEX, thancapturing of the at least two scaled sensor signals SC1, SC2 is stopped.The acquisition process 21 runs in real time. A data set DS generated bythe acquisition process 21 is used by the gesture message processing 22and the gesture event processing 23 to determine a gesture event GE. Thedata set DS of the acquisition process 21 are provided to the gesturegame processing 24 for determination of real time vectors.

The optical sensor arrangement 10 provides data from the at least twolight sensors 35, 36 comprising data that are proportional to theintensity of the at least two sensor signals S1, S2. The at least twosensor signals S1, S2 are processed to extract intensity changes. By thecrosstalk filters 20, the at least two sensor signals S1, S2 arefiltered and a DC component is subtracted.

In an alternative embodiment, the adaptive filters 18 and the crosstalkfilters 20 are realized as digital filters. In addition, the evaluationcircuit 12 comprises a summation unit 17′, a enter/exit thresholdsprocess unit 19′, an acquisition process unit 21′, a gesture messageprocessing unit 22′, a gesture event processing unit 23′ and a gesturegame processing unit 24′ which are implemented using logical gates orstate machines. Thus, the input 16 of the evaluation circuit 12 iscoupled via the summation unit 17′ to adaptive filters 18. An output ofthe adaptive filters 18′ is coupled to the enter/exit thresholds processunit 19′. The input 16 of the evaluation circuit 12 is coupled tocrosstalk filters 20. An output of the crosstalk filters 20 and anoutput of the enter/exit thresholds process unit 19′ are connected tothe acquisition process unit 21′. In addition, the output of theenter/exit thresholds process unit 19′ is also connected to the adaptivefilters 18. The acquisition process unit 21′ is connected to the gesturemessage processing unit 22′. The gesture event processing unit 23′follows the gesture message processing unit 22′. The output of theacquisition process unit 21′ is also connected to the gesture gameprocessing unit 24. The acquisition process unit 21′ is connected on itsoutput side to the crosstalk filters 20′.

FIG. 1B shows an exemplary embodiment of the optical sensor arrangement10 which is a further development of the optical sensor arrangement ofFIG. 1A. The optical sensor arrangement 10 comprises a light-emittingdiode 30, abbreviated LED. The LED 30 is for example realized as aninfrared light-emitting diode. Furthermore, the optical sensorarrangement 10 comprises a current source 31 for supplying the LED 30. Aseries connection of the LED 30 and the current source 31 is connectedbetween a supply voltage terminal 32 and a reference potential terminal33. The photodiode unit 13 and the LED 30 are arranged in such a waythat light emitted by the LED 30 which falls on an object 34 over theoptical sensor arrangement 10 is reflected to the photodiode unit 13.

The photodiode unit 13 comprises the first, the second, a third and afourth light sensor 35-38. The first to the fourth light sensor 35-38are implemented as photodiodes. The first to the fourth light sensor 35to 38 generate the first to the fourth sensor signals S1, S2, S3, S4. Amultiplexer 39 connects the output of the four light sensors 35 to 38 tothe ADC 14. The first light sensor 35 is sensitive in the northdirection N and the second light sensor 35 is sensitive in the southdirection S. Moreover, the third light sensor 37 is sensitive in thewest direction W and the fourth light sensor 38 is sensitive in the eastdirection E. The first to the fourth sensor signals S1-S4 are providedto the crosstalk filters 20. Furthermore, the first to the fourth sensorsignals S1-S4 are applied to the summation step 17. The first to thefourth sensor 35-38 detect light emitted by the LED 30 and reflected bythe object 34.

Silicon integration allows four light sensors 35-38 to be placed on asingle piece of silicon to detect light from different directions. Usingthis technology coupled with the single LED 30, the optical sensorarrangement 10 can detect motion over the device. To decode the lightreflection into motion requires processing as outlined above.

In the overall optical sensor arrangement 10, data comes from thephotodiodes 35-38 where the light is typically generated from an IR LED30 that is pulsed such that it is reflected from the object 34 overheadand provides background subtraction and A/D conversion. The outputs fromthis process are the four samples S1-S4 from the NSEW directions. Thisdata is then subsequently processed to extract the gesture information.

FIG. 1C shows an exemplary embodiment of a cross-section through theinput module 11. The cross-section goes through the first and the secondlight sensor 35, 36. The input module 11 is covered by a transparentoptical housing 40. The housing 40 has a lens 41 at its top. The firstand the second light sensor 35, 36 are sensitive in opposite directions.

FIG. 1D shows an exemplary embodiment of a characteristic of the foursensor signals S1 to S4 depending on the time t. When the object 34moves from the north to the south, the first sensor signal S1 has a peakthat arises before the peak of the second sensor signal S2. The thirdand the fourth sensor signals S3, S4 are nearly similar and have a peakbetween the peak of the first sensor signal S1 and the peak of thesecond sensor signal S2. The four peaks optionally have the same height.

A hand can be an example of the object 34. The sensitivity to thedirection provides different responses as the hand is passed over thefour light sensors 35-38. For example, if the hand is waved from northto south, the response would ideally look similar to FIG. 1D.

FIG. 2A shows an exemplary embodiment of the adaptive filters 18 and theenter/exit thresholds process 19. The adaptive filters 18 comprise aslow and a medium filter 50, 51. The slow and the medium filter 50, 51are realized by low pass filtering. The cutoff frequency of the slowfilter 50 is lower than the cutoff frequency of the medium filter 51. Aslow filter signal SFS and a medium filter signal SFM are generated fromthe summing signal SU by the slow and the medium filter 50, 51.Moreover, a minimum detector 52 of the adaptive filters 18 receives theslow filter signal SFS and the medium filter signal SFM. A baselinesignal SB is generated as a result of the minimum detector 52 that isprovided to the enter/exit thresholds process 19. The baseline signal SBis the minimum of the slow filter signal SFS and the medium filtersignal SFM. Moreover, the adaptive filters 18 comprise a fast averagingfilter 53 that receives the summing signal SU and provides an averagingfilter signal SFA to the medium filter 51. A signal representing a verylong gesture initializes the medium filter 51. A signal representing anactive gesture set a hold to the medium filter 51.

The fast averaging filter 53 may be realized by low pass filtering. Thecutoff frequency of the medium filter 51 may be lower than the cutofffrequency of the fast averaging filter 53.

Furthermore, the adaptive filters 18 comprise a noise filter 54 thatreceives the summing signal SU. The noise filter 54 is designed forcalculating the noise in the summing signal SU provided by the summationstep 17. The noise filter 54 is realized as a dynamic standard deviationfilter. A noise signal SN is generated out of the summing signals SU bythe noise filter 54.

The enter/exit thresholds process 19 performs a noise multiplying 55 ofthe noise signal SN and a standard deviation factor SDF. A distancesignal SD is generated by the noise multiplying 55. In the enter/exitthresholds process 19, the gesture entry threshold GEN is determined bya first summation 56 of the baseline signal SB and the distance signalSD. Furthermore, the exit gesture threshold GEX is determined by a firstmultiplying 58 of the distance signal SD with a first factor and by afirst adding 57 of the thus multiplied distance signal SD1 to thegesture entry threshold GEN. Moreover, the runtime gesture threshold RGTis determined by a second multiplying 60 of the distance signal SD witha second factor and by a second adding 59 of the thus multiplieddistance signal SD2 to the gesture exit threshold GEX. The first factoris, for example, 0.5, and the second factor is, for example, 2.5.

The minimum detector 52 generates the baseline signal SB whichrepresents a baseline of the summing signal SU. Thus, the adaptivefilters 18 can also be named a baseline filter. The distance signal SDis responsible for the distance between the gesture entry threshold GENand the baseline signal SB.

In such an optical sensor arrangement 10, a critical component is theconfiguration of the baseline filter 18 and algorithm for adjusting thebaseline signal SB due to dynamic system impact. This baseline signal SBis needed to determine when to start a gesture. An optional baselinearchitecture looks like FIG. 2A.

A decision regarding the presence of a gesture is typically made whenthe summing signal SU of the NSEW channels S1-S4, or any combination ofthose channels, exceeds a predetermined threshold such as the gestureentry threshold GEN. The baseline algorithm will provide an adaptivesystem such that when there is a change in the baseline, the gesturestart threshold GEN, that is also named gesture entry threshold, alsochanges. FIGS. 2G and 2H show an example of two gesture responses wherethe second response resulted in a baseline shift. Without an adaptivealgorithm, the optical sensor arrangement 10 would never come out of agesture mode. The baseline signal SB must be shifted upward toaccommodate this shift. Another example may be humidity slowly buildingon the glass and the case such that the baseline signal SB slowlyincreases. A further example of a problem condition is when the baselinesignal SB is too close to the gesture entry threshold GEN causingseveral invalid gestures to be generated as shown in FIGS. 2I and 2J.

The first step for generation of the gesture entry threshold GEN in theadaptive filters 18, also called baseline filter, is to direct theavailable information to the baseline system. The input to the system isthe NSEW data of the four sensor signals S1-S4 in any combination ofcomponents. Typically, this would be the sum of all four sensors signalsS1-S4, but it could also be the sum of a subset if determined that thiswould improve performance. For example, the first and the second sensorsignals 51, S2 could have a very high crosstalk and the third and thefourth sensor signals S3, S4 could have a much lower crosstalk. Thethird and the fourth sensor signals S3, S4 could be used alone for theinput in this situation. While there are many gesture feedbackconditions that could be considered, two of them will be handled asspecial conditions: a long gesture and a short gesture.

The basic algorithm for the baseline signal SB will be a dual filtersystem with the medium filter 51 and the slow filter 50. The adaptivefilters 18 are designed such that the baseline signal SB will increaseslowly but rapidly decrease or have a slow attach and a rapid decay.This can be implemented using the medium and the slow filter 50, 51 andselect the minimum of both filter signals SFM, SFS.

With the special condition of an excessively long gesture, a gestureengine comprising the adaptive filters 18 and the entry/exit thresholdsprocess 19 will never exit the gesture mode causing the optical sensorarrangement 10 to hand or repetitive gesture held conditions. In thiscase, the optical sensor arrangement 10 generates a long gesture abortand the feedback to the baseline signal SB would be to rapidly changethe baseline signal SB to a higher level. For the special condition ofexcessively short gestures, the difference signal SD, also namedthreshold offset, may need to be increased as this may be due toincreased system noise.

Referring to the noise filter 54, to develop the thresholds GEN, GEXabove the baseline signal SB, the noise in the incoming data must bedetermined. To do so, a standard deviation over a number of samples iscalculated. With this system, samples are not stored, so a dynamicstandard deviation filter is needed. Standard deviation is calculatedas:SN=Sqrt(Sum(A′*A′+B′*B′+C′*C′ . . . )),where A′=A−avg,B′=B−avg, C′=C−avg

The square root, abbreviated sqrt, is the last step, therefore, it willbe ignored for the time being, resulting in the equation:SN=Sum(A′*A′+B′*B′+C′*C′ . . . )

A, B and C are the values of the first, second and third sensor signalsS1, S2, S3. If an offset data such as offset-compensated sensor signalsSO1-SO4 are used, the average of A=B=C . . . will be zero. The sum canthen become a running sum of squares. With filtering, the followingfiltered standard deviation is provided:A=Current Sample,B=Last Filter OutputSN=Running Sum=A*A*0.1+B*B*0.9;

The current standard deviation is the square root of the running sum.The filter factor can be changed to adapt at an optimal rate.

Alternatively, a value D of the fourth sensor signal S4 is additionallyused for the calculation, using the equations:SN=Sqrt(Sum(A′*A′+B′*B′+C′*C′+D′*D′)) orSN=Sum(A′*A′+B′*B′+C′*C′+D′*D′) with D′=D−avg

The first to the fourth sensor signals S1, S2, S3, S4 are filtered bythe adaptive filters 18. The adaptive filters 18 are slow, medium and/orfast filters such as the first and the medium filter 50, 51 and the fastaveraging filter 53. The filters 50, 51, 53 are enabled by an inputsignal. The filters 50, 51, 53 are initialized by an input signal. Thefirst and the medium filter 50, 51 are initialized to the output of thefast averaging filter 53 which implements the fast filter. The adaptivefilters 18 comprise a standard deviation filter that is realized, forexample, as the noise filter 54. The standard deviation filter isenabled by an input signal. The standard deviation filter 54 is enabledby the slow filter stability.

The filters 50, 51, 53 are used to determine thresholds to enablefurther processing of the first to fourth sensor signals S1 to S4. Theslow and the medium filters 50, 51 are used in a multiple speedalgorithm. The slow filter 50 is used to increase the threshold such asthe gesture entry threshold GEN and the medium filter 51 is used todecrease the baseline SB. The standard deviation filter 54 is used incombination with the baseline signal SB to determine a threshold such asthe gesture entry threshold GEN. The noise signal SN can also be calleda standard deviation signal. The standard deviation filter is realizedas the noise filter 54. The standard deviation signal is determined fromthe standard deviation filter.

The noise signal SN is multiplied by a noise factor which is added tothe baseline. The noise multiplying 55 may be increased using anexternal signal. Alternatively, the noise multiplying 55 may bedecreased using an external signal. Thresholds are used to determine astart and stop for capturing data. For example, the gesture entrythreshold GEN determines a start for capturing data and the gesture exitthreshold GEX determines a stop for capturing data. “Gesture entrythreshold” may also be named a “gesture start threshold”. The gestureentry threshold GEN may be equal to the gesture exit threshold GEX.

In an alternative embodiment, the noise signal SN which represents thestandard deviation, is determined from a preset value. Thus, thestandard deviation filter 54 is omitted.

FIGS. 2B, 2C and 2D show exemplary embodiments of the slow and themedium filter 50, 51 and of the fast averaging filter 53. The basicconstruction of the three filters 50, 51, 53 is identical. As shown inFIG. 2B, the slow filter 50 comprises a first filter multiplier 61, afilter adder 62, a second filter multiplier 63 and a delay unit 64. Theinput of the slow filter 50 is coupled via the first filter multiplier61 to one input of the filter adder 62. The delay unit 64 couples anoutput of the filter adder 62 to the output of the slow filter 50. Theoutput of the slow filter 50 is coupled via the second filter multiplier63 to a second input of the filter adder 62. The first filter multiplier61 performs a multiplication by a first filter factor, whereas thesecond filter multiplier 63 is designed for multiplication with a secondfilter factor.

In the case of the slow filter 50, the first filter factor is, forexample, 0.001 and the second filter factor is, for example, 0.999.According to FIG. 2B, the medium filter 51 has for example a firstfilter factor 0.01 and a second filter factor 0.99. According to FIG.2D, the fast averaging filter 53 implements for example a first filterfactor 0.1 and a second filter factor 0.9.

One of the most important factors and parameters that need to beprogrammable are the time constants associated with the filters such asthe filters 50, 51, 53. The sample time of the ADC 14 will set the basictiming of the optical sensor arrangement 10 and when it changes, thefilter timing must also change. Therefore, the filter coefficients orfactors must be changed when the sampling time changes.

The selection of the time constants is highly dependent on humanfactors. One of the factors impacting time constant selection is thegesture vocabulary. If the vocabulary includes button pushes, longgestures are possible. If push buttons are not needed, then the timeconstants can be much faster. If push buttons are considered, then theremust be a decision regarding the longest button hold time beforeconsidering this an error condition and aborting the gesture. This timeconstant could be as long as 10 seconds. This is also impacted by thelongest time that the gesture engine could be stalled and not makingvalid gestures. In order to determine the right coefficients for thespecific time constant, the definition of time constant must also bestated. Another consideration is that the time constant must be muchslower than the slowest sampling time or else the filters 50, 51, 53 donot provide any value.

FIG. 2E shows an exemplary embodiment of a cross-section of the inputmodule 11, whereas a disturbing object 65 is on top of the housing 40.The disturbing object 65 influences the second light sensor 36 which issensitive in the south direction.

FIG. 2F shows an exemplary embodiment of signal characteristicsdepending on the time. Comparable to FIG. 1D, the peak of the firstsignal S1 is before the peak of the second sensor signal S2. However,the height of the peak of the second sensor signal S2 is lower than thepeak of the first sensor signal S1. The third and the fourth sensorsignals S3, S4 have peaks which can be detected between the peak of thefirst sensor signal S1 and the peak of the second sensor signal S2. Dueto the influence of the disturbing object 65, the peak of the thirdsensor signal S3 is lower than the peak of the fourth sensor signal S4.

The baseline will change due to many different factors. The opticalsensor arrangement 10 itself is optionally designed to have somepositive offset to ensure that accurate data is available even with noobject 34 present. Moreover, the optical sensor arrangement 10 istypically placed behind glass with ink that will cause some crosstalkand the ink will introduce part-to-part variation. Another change canoccur if the glass over the light sensors 35-38 is scratched. These aretypically permanent changes.

A further set of change may be semi-permanent or semi-static. A phonemay comprise the optical sensor arrangement 19. One example might beoperation in an airplane with the phone on the tray and the seatback infront of you leaned back. This condition would exist over a long periodbut eventually be removed. Another example could be if a person hadmake-up on and the make-up was transferred to the phone. This wouldremain until removed. Other more dynamic factors are when a person usesthe optical sensor arrangement 10 behind glass in a differentconfiguration. For example, if the optical sensor arrangement 10 is in acell phone, it could be used when a person is exercising and sweat couldcome in contact with the glass over the light sensors 35-38 causingdistortion. When the sweat is removed, the baseline signal SB willchange again. FIGS. 2E and 2F show an example of the response from aphone with a non-ideal baseline condition.

FIGS. 2G and 2H show exemplary embodiments of a signal characteristicdepending on the time t. As shown in FIG. 2G, the summing signal SU isgenerated by two gestures, whereas during the second gesture a baselineshift occurs. As shown in FIG. 2H, the baseline signal SB is amended bymeans of the adaptive filters 18. The dashed lines in FIGS. 2G and 2Hshow a value SB0 of the baseline signal SB and a value GEN0 of thegesture entry threshold GEN at the start of the first gesture. Thebaseline signal SB, and thus the gesture entry threshold GEN, areadapted by means of the adaptive filters 18 and the enter/exitthresholds process 19. The adaptation of the gesture entry threshold GENmay correct a drift of the sensor signals S1-S4. The drift may be causedby a changing surrounding. Thus, gestures may be better detected.

FIGS. 21 and 2J show an exemplary embodiment of a characteristic of thesumming signal SU over the time t. Whereas in FIG. 2I the differencesignal SD has a small value, the difference signal SD obtains a highervalue in the FIG. 2J. Due to the small value of the difference signal SDdisturbances in the summing signal SU result in two false gestures,shown in FIG. 2I. These false gestures can be avoided by a higher valueof the difference signal SD as shown in FIG. 2J.

FIG. 3A shows an exemplary embodiment of the crosstalk filters 20. Thecrosstalk filters 20 comprises four channels for processing the first tothe fourth sensor signal S1-S4. The first channel comprises a first slowfilter 70 and a first subtracting unit 71. The first sensor signal S1 isprocessed by the first channel of the crosstalk filters 20. A firstfiltered signal SF1 is generated by filtering the first sensor signal S1by the first slow filter 70. The first sensor signal S1 is directlyapplied to a non-subtracting input of the subtracting unit 71. The firstfiltered signal SF1 is fed to a subtracting input of the firstsubtracting unit 71. The first subtracting unit 71 outputs a firstoffset-compensated sensor signal SO1. Thus, the first offset-compensatedsensor signal SO1 is determined by subtracting the first filtered signalSF1 from the first sensor signals S1.

The other channels operate in a corresponding way. The crosstalk filters20 comprise a second, a third and a fourth slow filter 72, 73, 74 aswell as a second, a third and a fourth subtracting unit 75, 76, 77 whichare arranged corresponding to the first slow filter 70 and the firstsubtracting unit 71. Thus, the crosstalk filters 20 also generate asecond, a third and a fourth offset-compensated sensor signal SO2, SO3,SO4 by subtracting a second to fourth filtered signal SF2-SF4 from thesecond to the fourth sensor signal S2-S4. The slow filters 70, 72, 73,74 estimate the DC value of the first to the fourth sensor signal S1-S4.The first to the fourth slow filters 70, 72, 73, 74 perform a low-passfiltering. An enable signal which enables the operation of the filtersonly in case a gesture is not present, is provided to the first to thefourth slow filters 70, 72, 73, 74. The four scaled sensor signalsSC1-SC4 a determined by a DC-filtering of the four sensor signals S1-S4by the four slow filters 70, 72-74 and the four subtracting units 71,75-77.

In addition, the first channel of the crosstalk filters 20 comprise afirst crosstalk multiplier 78. The first offset-compensated sensorsignal SO1 and a first scale factor F1 is applied to an input of thefirst crosstalk multiplier 78. Thus, a first scaled sensor signal SC1 isgenerated by a multiplication of the first offset-compensated sensorsignal SO1 and the first scale factor F1 by the first crosstalkmultiplier 78. Correspondingly, the crosstalk filters 20 comprise asecond, a third and a fourth crosstalk multiplier 79, 80, 81. Thus, asecond, a third and a fourth scaled sensor signal SC2, SC3, SC4 aregenerated by a multiplication of the second, the third and the fourthoffset-compensated sensor signal SO2, SO3, SO4 with a second to a fourthscale factor F2-F4 using the second, third and fourth crosstalkmultiplier 79, 80, 81, respectively.

The crosstalk filters 20 perform a dynamic cancellation of the DC offsetand adjusting the scaling to equalize the different channels. FIG. 3Adescribes a DC offset and scaling architecture. The output S1-S4 fromthe data acquisition front end is filtered to produce an average valueSF1-SF4 which is subtracted from the data S1-S2 to remove the DCcomponent. This filter 20 has two components: a component which is fedback to the analog front end to ensure a slightly positive offset to thedata coming into the system and a second component 71, 75-77 which willsubtract the average data SF1-SF4 from the raw data S1-S4 to provide aDC offset data. The data SO1-SO4 must be slightly positive or else thevalue of that information is unknown since in most systems it is unknownhow far negative the raw data S1-S4 is below a zero value. Moreover, thefeedback to the analog front end is critical to eliminate any positiveoffset due to optical crosstalk. This increases the dynamic range of theoptical sensor arrangement 10.

Throughout the text, the term “data” may mean the four sensor signalsS1-S4 or signals derived from the four sensor signals S1-S4 such as thefour filtered signals SF1-SF4, the four offset-compensated sensorsignals SO1-SO4 or the four scaled sensor signals SC1-SC4 depending onthe context.

In an alternative not shown embodiment, the crosstalk filters 20comprise an offset register coupled to the non-subtracting input of thefirst subtracting unit 71. Alternatively, the offset register couplesthe first filter 70 to the first subtracting unit 71. The second to thefourth channel comprise corresponding registers. The programmable offsetregister allows adjustment of each channel's offset.

FIGS. 3B and 3C show exemplary characteristics of the first sensorsignal 51. In FIGS. 3B and 3C, conditions that could occur with positiveor negative offset are illustrated. The topmost curve in FIG. 3B issuitable for further processing, whereas the two other curves show anegative offset which has to be eliminated by the crosstalk filters 20.In FIG. 3C, the lowest curve can be used for further processing, whereasthe two other curves have a positive offset which is too high and whichhas to be reduced by the crosstalk filters 20. Optionally, theoffset-compensated signals SO1, SO2, SO3, SO4 have a small positiveoffset. The slow filters 70, 72, 73, 74 are designed in such a mannerthat the offset-compensated sensor signals SO1-SO4 are provided with asmall positive offset.

FIGS. 4A to 4C shown an alternative exemplary embodiment of thecrosstalk filters 20. For ease of understanding, only the first channelfor processing of the first sensor signal S1 is shown. The firstcrosstalk multiplier 78 as shown in FIG. 3A is omitted. Instead, theoffset compensated sensor signal SO1 is fed to a non-subtracting inputof a further first subtracting unit 82. A first scaling coefficient C1is provided to a subtracting input of the further first subtracting unit82. The first scaled sensor signal SC1 is determined by subtracting thefirst scaling coefficient C1 from the first offset-compensated sensorsignal SO1 by the further first subtracting unit 82. An offsetprocessing 83 is performed using the first filtered signal SF1. Theresult of the offset processing 83 is fed to the photodiode unit 13 andthus to the first light sensor 35. A target high value and a target lowvalue are used in the offset processing 83. The processing of the secondto the fourth sensor signal S2 to S4 occurs correspondingly using asecond to a fourth scaling coefficient C2-C4. The filtered signalsSF1-SF4 are used to determine the target value for the analog frontend.The analog frontend comprises the input module 11. The target isincreased if the value is below a low target value. The target isdecreased if the value is above a high target value. Overall offset isan issue, but channel to channel offset differences also cause problems.The crosstalk filters 20 solve the problem with different offsets todifferent channels.

FIG. 4B shows an exemplary embodiment of the first slow filter 70. Thesecond to the fourth slow filter 72-74 operate such as the first slowfilter 70. The first slow filter 70 is constructed in a similar way asthe slow filter 50 that is embedded in the adaptive filters 18. Thefirst slow filter 70 comprises a first multiplier 84, an adder 85, adelay unit 86 and a second multiplier 87. The first sensor signal S1 ismultiplied by the first multiplier 84 such that the multiplied sensorsignal is applied to an input of the adder 85. An output of the adder 85is coupled via the delay circuit 86 to an output of the first slowfilter 70. The output of the first slow filter 70 is coupled via thesecond multiplier 87 to a further input of the adder 85. A firstmultiplying factor, e.g. 0.001, is applied to the first multiplier 84,whereas a second multiplying factor, such as 0.999, is applied to thesecond multiply unit 87.

Some of the most important factors and parameters that need to beprogrammable, are the time constants associated with the slow filters70, 72-74. The sample time of the ADC 14 sets the basic timing of theoptical sensor arrangement 10, and it must also change when the filtertiming changes. Therefore, the filter coefficients must be changed whenthe sampling time changes.

FIG. 4C shows an exemplary characteristic of the first to the fourthscaled sensor signals SC1-SC4. All four scaled sensor signals SC1 to SC4arise from the same baseline and obtain the same peak value.

FIG. 5A shows an exemplary alternative embodiment of the optical sensorarrangement 10. The input module 11 does not comprise the memory 15. TheADC 14 is directly connected via the input 16 of the evaluation circuit12 to the crosstalk filters 20. The crosstalk filters 20 comprise a DCoffset removal process 90. The DC offset removal process 90 can beimplemented using the first to the fourth slow filter 70, 72 to 74 andthe first to the fourth subtracting units 70, 75 to 77. The DC offsetremoval process 90 provides a first offset-compensated sensor signal S01to the further first subtraction unit 82. The first scaling coefficientC1 is provided to the subtracting input of the first further subtractionunit 82. Whereas in FIG. 4A the first scaling coefficient C1 has apredetermined value, the first scaling coefficient C1 is variable inFIG. 5A.

The first offset-compensated sensor signal SO1 is processed by a sampleddata process 91, a normalize data process 92 and a coefficient filter93. The first scaling coefficient C1 is calculated by sampling the firstoffset-compensated sensor signal SO1, normalizing the signal andfiltering the signal. Advantageously, the first scaling coefficient C1is time-dependent and reduces the influence of the surroundings of theoptical sensor arrangement 10. The second to the fourth scalingcoefficients C2-C4 are generated in a corresponding manner.

The first to the fourth scaled sensor signals SC1-SC4 are provided tothe acquisition unit 21. The first to the fourth scaled sensor signalsSC1-SC4 are processed by a peak detection 95. Furthermore, the result ofthe peak detection 95 and the four scaled sensor signals SC1-SC4 areprocessed by a data qualification 89. A result of the data qualification89 is used for the sampling of the offset-compensated sensor signalsSO1-SO4. Thus, in FIG. 5B, an alternative DC offset and scalingarchitecture is provided.

The filtered data such as the first to the fourth sensor signals S1 toS4 are part of a gesture optical sensor arrangement 10. The gesture isstarted when the data is above the gesture start threshold GEN. Thegesture is ended when the data falls below the gesture stop thresholdGEX. The scaled filtered data, such as the first to the fourth scaledsensor signals SC1-SC4, is processed by the gesture optical sensorarrangement 10, for example by the gesture message processing 22. Thegesture optical sensor arrangement 10 provides feedback to furtherqualify the sample point. The gesture optical sensor arrangement 10provides feedback to recalibrate the optical sensor arrangement 10.Coefficients, such as the first to the fourth scaling coefficient C1-C4or the first to the fourth scale factors F1-F4, are stored in along-term memory and are initialized during the next gesture systemstart. The filtered signals SF1 to SF4 are further processed todetermine the maximum value of the sensor signals between the start andend indication. The peak data point is used to capture the raw data atsaid peak and/or the offset data at said peak. The peak data isqualified with some system parameters. The system qualificationparameters are above a threshold and/or are below a maximum threshold.The peak data values are normalized to provide scaling factors such asthe scaling factors F1-F4 provided to the crosstalk multipliers 78 to81. The normalized scaling factors F1-F4 are calculated by filtering.The filtered scaling factors are used to scale the raw or offset data.The data is acquired in a register; the maximum point and the number ofaccumulated points are processed.

FIG. 5B shows an exemplary embodiment of the acquisition process 21. Inthe acquisition process 21, an analyze gesture process 100 determinesthe data set DS having the information such as “long gesture”, “goodgesture”, “abort gesture”, and/or “runtime/short gesture”. For thispurpose, a data accumulation step 96 is performed using the first to thefourth scaled sensor signals SC1-SC4. A signal indicating an activegesture is used in the data accumulation step 96. The result of the dataaccumulation step 96 is stored in a memory 99. Moreover, a maximum indexstep 98 is performed using the first to the fourth scaled sensor signalsSC1-SC4. The analyzed gesture process 100 uses the content of the memory99, the result of a counter 97, the result of the maximum index step 98and a signal indicating the runtime gesture threshold RGT. The data setDS indicating a long gesture, a good gesture, an abort gesture and/or aruntime/short gesture is fed to the gesture message processing 22.

The data processing determines if the number of samples is below athreshold indicating a short gesture. The data processing determines ifthe maximum level of the gesture falls below a predetermined threshold.The data processing determines if the gesture length exceeds apredetermined threshold. Moreover, the data processing determines if thegesture is a normal good gesture. The runtime gestures are furtherprocessed to determine if the number of invalid runtime gestures exceedsthe number of acceptable runtime gestures over a period of time. Theshort gestures are further processed to determine if the number ofinvalid short gestures exceeds the number of acceptable short gesturesover a period of time.

The optical sensor arrangement 10 is sensitive in four directionalcomponents: north, south, east and west. The sensor signals are usedwith data proportional to the intensity of the input. The data is partof a system with input indicating the start and end of data acquisition.The four directions of data are acquired and stored in a memorysubsystem.

FIG. 5C shows an exemplary embodiment of a sensor response to the fourdirections. The sensor response can be written as a vector. The firstcoordinate of the vector is the north to south strength NSs, abbreviatedN−S strength. The N−S strength NSs can be calculated according to thefollowing equation:NSs=(N−S)/(N+S)

The second coordinate of the vector is the east to west strength EWs,abbreviated E−W strength. The E−W strength EWs can be calculatedaccording to the following equation:EWs=(E−W)/(E+W)

The first value N depends on the first sensor signal S1 or on a signalderived from the first sensor signal S1 such as the first scaled sensorsignal SC1 or the first offset-compensated sensor signal SO1.Correspondingly, the second value S depends on the second sensor signalS2 or on a signal derived from the second sensor signal S2 such as thesecond scaled sensor signal SC2 or the second offset-compensated sensorsignal SO2.

Similarly, the third value W depends on the third sensor signal S3 or ona signal derived from the third sensor signal S3 such as the thirdscaled sensor signal SC3 or the third offset-compensated sensor signalSO3. The fourth value E depends on the fourth sensor signal S4 or on asignal derived from the fourth sensor signal S4 such as the fourthscaled sensor signal SC4 or the fourth offset-compensated sensor signalSO4.

As can be seen in FIG. 5C, a movement of the object 34 from the north tothe south leads to a positive N−S strength and an east to west strengthwhich is nearly zero. A movement of the object 34 from the south to thenorth results in a negative N−S strength and an E−W strength ofapproximately zero. Correspondingly, a movement of the object 34 fromthe west to the east leads to an N−S strength of approximately zero andto an E−W strength that is negative. A movement of the object 34 fromthe east to the west results in an N−S strength that is approximatelyzero and a positive E−W strength. The object 34 can, for example, be aperson's hand.

Thus, there is a direct mapping from the normalized N−S strength and thenormalized E−W strength to provide a vector indicating the position ofthe hand relative to the optical sensor arrangement 10, for exampleabove the optical sensor arrangement 10. The N−S strength and the E−Wstrength are translated to polar coordinates giving a direction andmagnitude for the vector.

If the optical sensor arrangement 10 only needs two of the fourdirections, the other two directions can be forced to zero giving twodirections plus the magnitude. The magnitude and direction are:Dir=arctan(NSs/EWs),Mag=sqrt(NSs*NSs+EWs*EWs)

There is a direct mapping from the normalized N−S strength and thenormalized E−W strength to provide a vector indicating the position ofthe hand relative to the optical sensor arrangement 10, for exampleabove the arrangement 10.

FIG. 6A shows an exemplary embodiment of the gesture message processing22. As a result the gesture message processing 22 determines a gesturemessage GM using the data set DS that is also named a good gesture dataset. The data set DS may, for example, be provided by the memory 99. Thedata set DS is processed by a gesture filtering 101 and a search step102 for an entry/exit vector index. The result of the search step 102 isused in an entry vector process 103 and an exit vector process 104.Moreover, the gesture message processing 22 comprises a magnitudeprocess 105 which uses a maximum index. The gesture message processing22 comprises a time stamp 106. The gesture message GM results from theentry vector process 103, the exit vector process 104, the magnitudeprocess 105 and the timestamp 106.

FIG. 6B shows an exemplary embodiment of the gesture event processing23. The gesture message GM is further processed by the gesture eventprocessing 23 resulting in the gesture event GE. For this purpose, thegesture message GM is used in a north south process 107, in an east westprocess 108, in a north south east west process 109, in a diagonalsprocess 110 and in an entry/exit process 111. The gesture event GE isdetermined by selecting a result of these five processes 107 to 111 by amultiplexing process 112 which is controlled by a type signal ST.

FIG. 6C shows an exemplary embodiment of the north south east westprocess 109, abbreviated NSEW process. The NSEW process 109 uses thegesture message GM for providing the gesture event GE with theinformation whether the gesture is a tap, a button or a cross. The tapis detected by a check tap step 113 and a check direction step 114. Thebutton is received by a check button step 115. The cross is derived by afurther check direction step 116.

The optical sensor arrangement 10 is designed to operate according to aprocessing method. The processing method comprises a single point for ashort gesture. The processing method may search for a percentage of thepeak value. The entry is searched from the beginning of a gesture. Theexit is searched starting from the end of the gesture. The data isaveraged to reduce noise.

The optical sensor arrangement 10 is sensitive in four shadowcomponents, such as shadow north, shadow south, shadow east and shadowwest. The optical sensor arrangement 10 provides data from the fourlight sensors 35-38 comprising data proportional to the intensity of theinput. The data is part of a system with input indicated the start andend of data acquisition. The four direction of data are acquired andstored in a memory subsystem, such as the memory 99. The shadownorth—shadow south directions are processed to generate a differentialvalue N−S. The shadow east—shadow west direction are processed togenerate a differential value E−W. The N−S and E−W data is normalized bydividing by (N+S) or (E+W). The N−S and E−W data is normalized bydividing by (N+S+E+W)/2. The shadow north—shadow south directions areprocessed to generate a differential value (N−S). The shadow east—shadowwest direction are processed to generate a differential value (E−W). TheN−S and E−W data is used to form a vector and magnitude indicating thedirection of the data.

The entry and exit vectors ENV, EXV are processed against N/S criteriato provide a N−S or S−N gesture, against E/W criteria to provide a E−Wor W−E gesture, against NSEW criteria to provide a N−S or S−N or E−W orW−E gesture, against 8 direction criteria to provide a N−S or S−N or E−Wor W−E or NE−SW or SE-NW or NW−SE or SW−NE gesture, against NSEWcriteria to provide a TapN or a TapS or a TapE or a TapW gesture, and/oragainst NSEW criteria to provide a ButtonN or a ButtonS or a ButtonE ora ButtonW gesture.

The entry and exit vectors ENV, EXV are processed to determine a varietyof gestures. The entry and exit vectors ENV, EXV are processed todetermine if they are in opposite direction. The opposite vectors areprocessed to determine the crossing direction. The crossing direction isquantized and translated into a specific gesture. The specific gesturemay be N−S or S−N. The specific gesture may be E−W or W−E. The specificgesture may be NE−SW or SW−NE. The specific gesture may be NW−SE orSE−NW. There may be also other levels of quantizing forming specificgesture.

The entry and exit vectors ENV, EXV are processed to determine if theyare in the same direction. The direction is quantized and translatedinto a specific tap gesture. The specific gesture may be a tap N, a tapS, a tap E, a tap W and/or a tap N. The specific gesture may be anotherlevel of quantizing forming a specific gesture.

The entry vector ENV is coupled with a time exceeding a specific limit.The entry vector ENV is processed to determine an entry direction. Theentry direction is quantized forming a specific gesture. The specificgesture is an N button push, an S button push, an E button push, and/ora W button push. The specific gesture may be another level of quantizingforming a specific gesture.

A length of the gesture may be less than a fixed limit such as a firstpredetermined duration. The gesture event GE is a tap event.

A length of the gesture may exceed a fixed limit such as a secondpredetermined duration. After that limit a button event is declared asthe gesture event GE. The length is continually monitored to determinecontinuing events. After that limit a button pushed event is declared.After an incremental period of time a button held event is declared.After the gesture ends, a button released event is declared. In thegesture message processing 22 the data is collected and processed in theoptical sensor arrangement 10 as shown in the plots and graphicalpictures of FIGS. 6D to 6K.

FIGS. 6D and 6E show exemplary embodiments of signal characteristicsdepending on the time t. In FIG. 6D, a signal SENV indicates an envelopeof the NSEW signals. The signal SENV represents the envelope of thefirst to the fourth sensor signal S1-S4. A signal NSs indicates the N−Sstrength and a signal EWs indicates the E−W strength. A signal SENV1shows the envelope of an absolute value of the S−N strength and of theabsolute of the E−W strength. A push button is calibrated. The followingsteps are performed: A search from the left for an entry peak of theSENV signal. A search from the right for an exit peak of the envelopesignal SENV. A search from the left to the peak for an entrydifferential peak. A search from the right to the peak for an exitdifferential peak. Then data are taken at different peaks and an entryvector ENV and an exit vector EXV are calculated.

In FIG. 6E, an angle signal SANG of the N−S strength and the E−Wstrength is depicted with a consistent entry and exit angle. The nextstep is to determine the input and output sampling points. Manydifferent gestures are taken into account: short gestures, longgestures, long distance gestures, gestures with a flat hand, gestureswith the side of a hand, gestures with the flat hand and with fingersapart etc. The optimal correct points are chosen by searching from thebeginning to the maximum value to determine the peak differentialmagnitude and using this point as the sample point. This provides themaximum signal to noise ratio. For the case where the gesture has a fewnumber of points, the optical sensor arrangement 10 selects a fixedpoint position to determine the entry and exit point. For example, if 5points are sampled, the center could be point 3, then point 2 could betaken as the entry point and point 4 as the exit point.

An alternative method is to take some percentage of the maximum valueand search the envelope of total gesture power for that point startingfrom the beginning and from the ending. For example, if there were 21points in the gesture and the maximum value was 200, a threshold of ½ ofthe maximum could be used for the entry and exit point. Searching andselecting a point around that threshold can be used.

An alternative method for a gesture with a larger number of points wouldbe to average the points before extracting the entry and exit vectorsENV, EXV. This is especially useful for low amplitude, for example at along distance gesture. Another method would be a mixture of the abovetwo methods where the optical sensor arrangement 10 searches for thehalf of the mid-point of the average of a couple of points using aweighted filter. This method minimized the amount is signal processingneeded. An overall algorithm could use a combination of the above.

Exemplary data from the gesture message processing 22 is shown below:

Gesture

Entry 35.09@73 deg

Exit 36@303 deg

Duration 58

Repetition −2147483648

nDiff Strength 0.98

nN−S Strength 0.19

nE−W Strength 0.97

nNE-SW Strength 0.55

nNW−SE Strength 0.79

dDiff Strength 77.25

dN−S Strength −72.00

dE−W Strength −28.00

dNE-SW Strength −42.00

dNW−SE Strength −30.00

Prox4 529.00

* d—Differential

** n—Normalized

The gesture message GM provides all of the information needed todetermine the gesture type.

FIGS. 6F, 6G and 6H show an exemplary embodiment of the interpretationof one movement. Whereas FIG. 6F illustrates a north to south system,FIG. 6G illustrates a west to east system. The same movement of theobject 34 occurs in FIGS. 6F, 6G and 6H. For the north to south systemof FIG. 6F the movement is interpreted as an N−S gesture. In thewest-east system of FIG. 6G the same movement is determined as a W−Egesture. FIG. 6H shows a north-east-west-south system.

The decision process for the NS vectors is to determine if the entryvector ENV is between −90 and +90 degree, then the entry was from thenorth. If the exit vector EXV is between 90 and 270 degree, then theexit is to the south and it is classified as an N−S vector. FIGS. 6F and6G show an illustration of the NS and EW processing. For the givengesture, it would be interpreted as an N−S gesture in the NS mode and aW−E gesture in the EW mode. In the four direction NSEW mode of FIG. 6H,the decision point must change to 45 degree angles. The previous gesturewould be interpreted with the following plot: From the plot, it is not aclear N−S or W−E gesture. The gesture enters from the W and exits to theS. In order to make a decision, more information is needed. In thiscase, the magnitude of the N−S and E−W are compared and the largestmagnitude is selected as the dominate gesture and that result selected.For example, if the entry W−E vector is larger than the exit N−S vector,the gesture is determined to be a W−E gesture.

FIG. 6I shows an exemplary embodiment of a west tap gesture. For alldirection modes, taps can also be enabled provided the entry and exitvectors ENV, EXV are the same. Push buttons are also provided in eachmode where there is an entry vector ENV and there is a long gesture. Forthe tap gesture, the entry vector ENV and the exit vector EXV must be inthe same direction. The following is an example of a West tap gesture.

FIG. 6J shows an exemplary embodiment of a table of gesture events GE.In FIG. 6J, an example of gesture events GE showing the five possibleevent types is given. A cross_N_S gesture is realized by a movement ofthe object 34 from the north to the south. A cross_NE_SW gesture isimplemented by a movement of the object 34 from the northeast to thesouthwest. A tap N gesture is realized by a short gesture, when theobject 34 is moved to the north of the optical sensor arrangement 10. Abutton_N gesture is determined, when the object 34 moves to the north ofthe optical sensor arrangement 10 and a long gesture occurs. Othergestures are a start gesture, a button_held gesture and a button_releasegesture.

Alternative sets of possible gestures events GE could comprise:

-   2 Directions: N−S, S−N-   2 Directions: E−W, W−E-   4 Directions: N−S, S−N, E−W, W−E-   8 Directions: N−S, S−N, E−W, W−E, NE−SW, SW−NE, NW−SE, SE−NW-   EE Event: Entry and Exit Vector data available

FIG. 6K illustrates exemplary embodiments of gestures from the left tothe right such as a north to south gesture, a north button gesture, anorth tap gesture and a button held gesture.

The invention claimed is:
 1. An optical sensor arrangement, comprising:at least two light sensors for providing at least two sensor signals;and an evaluation circuit that comprises an input coupled to the atleast two light sensors and is designed to generate at least twofiltered signals by filtering the at least two sensor signals, togenerate at least two offset-compensated sensor signals by subtractingthe filtered signals from the corresponding sensor signals and togenerate an entry vector indicating the position of an object relativeto the at least two light sensors at the start of a gesture and an exitvector indicating the position of the object relative to the at leasttwo light sensors at the end of the gesture, wherein a first coordinateof a vector is a north to south strength and a second coordinate of thevector is an east to west strength, and wherein the first and the secondcoordinates of the vector depend on the at least two offset-compensatedsensor signals.
 2. The optical sensor arrangement according to claim 1,wherein the evaluation circuit is designed to recognize a tap gesture,in response to a determination that the entry vector and the exit vectorare nearly the same and the duration of the gesture is less than a firstpredetermined duration.
 3. The optical sensor arrangement according toclaim 1, wherein the evaluation circuit is designed to recognize abutton gesture, in response to a determination that the entry vector andthe exit vector are nearly the same and the duration of the gesture islarger than a second predetermined duration.
 4. The optical sensorarrangement according to claim 1, wherein the evaluation circuit isdesigned to start capturing the at least two sensor signals for gesturedetection, in response to a determination that a signal derived from theat least two sensor signals is larger than a gesture entry threshold. 5.The optical sensor arrangement according to claim 4, wherein theevaluation circuit is designed to dynamically adapt the gesture entrythreshold depending on the previous values of the at least two sensorsignals.
 6. The optical sensor arrangement according to claim 4, whereinthe evaluation circuit is designed to determine the gesture entrythreshold by adding a distance signal to a baseline signal.
 7. Anoptical sensor arrangement, comprising: at least two light sensors forproviding at least two sensor signals; and an evaluation circuit thatcomprises an input coupled to the at least two light sensors and isdesigned to evaluate the at least two sensor signals for gesturedetection in an adaptive manner using previous values of the at leasttwo sensor signals, wherein the evaluation circuit is designed to startcapturing the at least two sensor signals for gesture detection, inresponse to a determination that a signal derived from the at least twosensor signals is larger than a gesture entry threshold, dynamicallyadapt the gesture entry threshold depending on the previous values ofthe at least two sensor signals, such that the evaluation circuit isdesigned to determine the gesture entry threshold by adding a distancesignal to a baseline signal, generate at least two filtered signals byfiltering the at least two sensor signals, and generate at least twooffset-compensated sensor signals by subtracting the filtered signalsfrom the corresponding sensor signals.
 8. The optical sensor arrangementaccording to claim 7, wherein the evaluation circuit is designed todetermine a noise signal by noise filtering of a sum of the at least twosensor signals, and the distance signal by multiplying a standarddeviation factor with the noise signal.
 9. The optical sensorarrangement according to claim 7, wherein the evaluation circuit isdesigned to generate at least two scaled sensor signals by multiplyingthe at least two offset-compensated sensor signals with at least twoscale factors, or by adding the at least two offset-compensated sensorsignals and at least two scaling coefficients.
 10. An optical sensorarrangement, comprising: at least two light sensors for providing atleast two sensor signals, and an evaluation circuit that comprises aninput coupled to the at least two light sensors and is designed toevaluate the at least two sensor signals for gesture detection in anadaptive manner using previous values of the at least two sensorsignals, wherein the evaluation circuit is designed to start capturingthe at least two sensor signals for gesture detection, in response to adetermination that a signal derived from the at least two sensor signalsis larger than a gesture entry threshold, dynamically adapt the gestureentry threshold depending on the previous values of the at least twosensor signals, such that the evaluation circuit is designed todetermine the gesture entry threshold by adding a distance signal to abaseline signal, determine a slow filter signal by filtering a sum ofthe at least two sensor signals with a slow filter, determine a mediumfilter signal by filtering the sum of the at least two sensor signalswith a medium filter, and determine the baseline signal by selecting theminimum of the slow filter signal and the medium filter signal, the slowand the medium filter being realized by low pass filtering and a cutofffrequency of the slow filter is lower than a cutoff frequency of themedium filter.
 11. A method for gesture detection, comprising: providingat least two sensor signals by at least two light sensors; generating atleast two filtered signals by filtering the at least two sensor signals;generating at least two offset-compensated sensor signals by subtractingthe filtered signals from the corresponding sensor signals; andgenerating an entry vector indicating the position of an object relativeto the at least two light sensors at the start of a gesture and an exitvector indicating the position of the object relative to the at leasttwo light sensors at the end of the gesture, wherein a first coordinateof a vector is a north to south strength and a second coordinate of thevector is an east to west strength, and wherein the first and the secondcoordinates of the vector depend on the at least two offset-compensatedsensor signals.
 12. A method for gesture detection, comprising:providing at least two sensor signals by at least two light sensors;evaluating the at least two sensor signals for gesture detection in anadaptive manner using previous values of the at least two sensorsignals, wherein capturing the at least two sensor signals for gesturedetection starts, in response to a determination that a signal derivedfrom the at least two sensor signals is larger than a gesture entrythreshold, and the gesture entry threshold is dynamically adapteddepending on the previous values of the at least two sensor signals suchthat the gesture entry threshold is determined by adding a distancesignal to a baseline signal; generating at least two filtered signals byfiltering the at least two sensor signals; and generating at least twooffset-compensated sensor signals by subtracting the filtered signalsfrom the corresponding sensor signals.
 13. A method for gesturedetection, comprising: providing at least two sensor signals by at leasttwo light sensors; evaluating the at least two sensor signals forgesture detection in an adaptive manner using previous values of the atleast two sensor signals, wherein capturing the at least two sensorsignals for gesture detection starts, in response to a determinationthat a signal derived from the at least two sensor signals is largerthan a gesture entry threshold, and the gesture entry threshold isdynamically adapted depending on the previous values of the at least twosensor signals such that the gesture entry threshold is determined byadding a distance signal to a baseline signal; determining a slow filtersignal by filtering a sum of the at least two sensor signals with a slowfilter; determining a medium filter signal by filtering the sum of theat least two sensor signals with a medium filter; and determining thebaseline signal by selecting the minimum of the slow filter signal andthe medium filter signal, wherein the slow and the medium filter arerealized by low pass filtering and a cutoff frequency of the slow filteris lower than a cutoff frequency of the medium filter.